Papers with fluency evaluation metrics
SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Visual captioning is an open-ended area for evaluation, requiring specialized training to improve human-correlation. |
| Approach: | They propose a new evaluation framework rooted in information theory . they propose metric SPURTS and metric SMURF to measure fluency . |
| Outcome: | The proposed metrics achieve state-of-the-art correlation with human judgment compared with other evaluation metrics. |